Improving the Performance of Complex Agent Plans Through Reinforcement Learning

Agent programming in complex, partially observable and stochastic domains usually requires a great deal of understanding of both the domain and the task, in order to provide the agent with the knowledge necessary to act effectively. While symbolic methods allow the designer to specify declarative knowledge about the domain, the resulting plan can be brittle since it is difficult to supply a symbolic model that is accurate enough to foresee all possible events in complex environments, especially in the case of partial observability. Reinforcement Learning (RL) techniques, on the other hand, can learn a policy and make use of a learned model, but it is difficult to reduce and shape the scope of the learning algorithm by exploiting a priori information. We propose a methodology for writing complex agent programs that can be effectively improved through experience. We show how to derive a stochastic process from a partial specification of the plan, so that the latter's perfomance can be improved solving a RL problem much smaller than classical RL formulations. Finally, we demonstrate our approach in the context of Keepaway Soccer, a common RL benchmark based on a RoboCup Soccer 2D simulator. Copyright © 2010, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved

Local boundedness of vectorial minimizers of non-convex functionals

We prove a local boundedness result for local minimizers of a class of non-convex functionals, under special structure assumptions on the energy density. The proof follows the lines of that in [CupLeoMas17], where a similar result is proved under slightly stronger assumptions on the energy density

Limitatezza locale di minimi vettoriali di funzionali non convessi

We prove a local boundedness result for local minimizers of a class of non-convex functionals, under special structure assumptions on the energy density. The proof follows the lines of that in [CupLeoMas17], where a similar result is proved under slightly stronger assumptions on the energy density.Dimostriamo un risultato di limitatezza locale per minimi locali di una classe di funzionali non convessi, con particolari ipotesi di struttura sulla densità di energia. La dimostrazione procede come quella in [CupLeoMas17], dove un risultato simile è dimostrato con ipotesi leggermente più forti sulla densità di energia

Learning Finite State Controllers from Simulation

Abstract. We propose a methodology to automatically generate agent controllers, represented as state machines, to act in partially observable environments. We define a multi-step process, in which increasingly accurate models- generally too complex to be used for planning- are employed to generate possible traces of execution by simulation. Those traces are then utilized to induce a state machine, that represents all reasonable behaviors, given the approximate models and planners previously used. The state machine will have multiple possible choices in some of its states. Those states are choice points, and we defer the learning of those choices to the deployment of the agent in the real environment. The controller obtained can therefore adapt to the actual environment, limiting the search space in a sensible way